Implementation and Evaluation of Four Different Methods of Negation Detection

Negation status identification for findings or diagnoses is an important medical data mining problem. Negative qualifier assigned to a medical condition may indicate the absence of the condition, so the ability to reliably identify the negation status of medical concepts affects the quality of results produced by the indexing and search tools. Searching for the best negation algorithm to use in our negation module for the suite of NLP tools, we modified two existing regular expression-based algorithms in an attempt to improve their performance, and created two classification-based methods. The classification-based algorithms were trained on 1745 discharge reports from a Bostonbased hospital. The algorithms were evaluated on 100 randomly-chosen outpatient reports from two different Boston-based hospitals, and the results were compared to the gold standard created by two independent human reviewers. The regular expression and syntactic processingbased algorithms appeared to have better agreement (Kappa = 0.77 to 0.79) with the human reviewers than the classification-based algorithms (Kappa = 0.57 to 0.75). The accuracy of regular expressions methods (91.9-92.3%) was also higher than that of classification based methods (83.5-89.9%). Based on our results, we have selected NegEx algorithm for our negation module.